Agenda 



Introduction 
Why are we fuzzing? 
Types of existing fuzzers 
Fuzzing, process 
Adoption Risks 
Fuzzing costs 
Pulling it all together 



All about the bugs! 



...Or really Bug Cost... 

Fuzzing is about finding bugs 

Fuzzing is repeatable 

Fuzzing *should* be easy on the wallet 

° Cost per Bug 
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...open source fuzzers... 
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Actively Maintained 
Bug Fixes Only 
Unknown 
Un-Maintained, but used 



open source fuzzers 



Commercial Fuzzers 



Mu Dynamics (aka Mu Security) 

° Network only! 

beSTORM 
D General 

Codenomicon 

D The general fuzzer that isn't a fuzzer 



One-off fuzzers 



Dom-Hanoi 
Hamachi 
Mangleme 
AxMan 



Sometimes needed but... 
Where are the mutations!? 



The Process 



Investigate 

Modeling 

Validate 

Monitor 

Run 

Results 



Modeling 



Model data of our system 

° Data Types 

° Relationships (size, count, offset) 

° Etc. 

Model state of our system 

° Send, Receive, Call, etc. 



Most of your time is spent here 

Unless a model already exists! 



Modeling 



Large difference between fuzzers 

n Language (Code vs. XML vs. Custom) 

n Extent of modeling allowed 

D Tools 

GUITools 

Format -> Model converters 



Modeling Examples 



Peach -XML 



< Data Mod el name="Example"> 
<Number size="8" signed="true"> 
<Relation type="size" of="Name , 7> 
</Number> 
<String name="Name" value="John Doe" /> 

</DataModel> 



Modeling Examples 



Sulley-Python/SPIKE 



s_size("Name" / length=i, fuzzable=True) 
if s_block_start("Name"): 

s_string("John Doe") 
s_block_end() 



Monitor 



Basic monitoring: 

n Debugger 

n Network capture 



Advanced monitoring 

n Easily pluggable 
° VM Control 
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Parallel Runs 



Single iteration from 5 to 60 seconds or more 
Target iterations: 250,000 -> 500,000 

n 500,000 tests * 30 seconds/test = 174 days! 
D Parallel by 10 = 17 days 
D Parallel by 20 = 9 days 

Run across multiple machines 

n Entry: 10 to 100 

n Advanced: 100 to 10,000+ 



Results 



Time intensive to sort hundreds of crashes 
Many crashes not interesting 
Many crashes are duplicates 



Crash Analysis!! 



Crash Analysis 



Bucketing of duplicate crashes 

° Hundreds to thousands of duplicates 

Analysis of exploitability 



Microsoft's lexploitable for WinDbg 
° Peach 



D ??? 



Sustainability 



How many years has tool existed? 

When was last release? 

Does project have commercial backing 

How many active leaders? 

Active community? 

n Forums, mailing lists, etc. 



Sustainability 



Current Last Release Years Commercial Active 
Version Date Available Community 
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Codenomicon 



2009 



Yes 



beSTORM 



2008 



Yes 



Usability 



...possibly Maturity? 

Documented? 

Online support forums? Do people answer 
questions? 

Publications? (e.g. books) 

Are external users a priority? 
° Vs. Internal tool released publicly 



Support & Training 



Training 

n Get staff going fast 

n Taking it to next level 

Support 
n Bugs, etc. 
D Assistance 



License Restrictions 



GPL 

° Must release changes 

° Taint issues? 

MIT 

° No restrictions 

BSD 

° No restrictions 

Commercial 
Should be okay for use 



Adoption Risks 



Sustainability Usability Training Support License 
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Time Spent in Order 



1. Modeling 

n Data & State, aka Creating a Definition 

2. Monitoring 

D Debugger Collection 

° Network capture (or other) 

D Restarting fuzzer 

3. Crash Analysis 

n Is it exploitable? 
Is it a duplicate? 



Hidden Costs 



Ramp-upTime 
Modeling 
Crash Analysis 



Paying to avoid these 

n But. ..custom formats/protocols 



SDL 



Fuzzing as part of SDL widely different from 
Research fuzzing. Companies have limited 
budget, resources, and time frame. 

D Need crash analysis 

Need integrated monitoring of target 
D Need parallel run ability (for Smart) 



Open vs. Commercial 



Fuzzing definitions (grammars) 

Training 

Support 

Consulting Services 

"Easy to use" 



Q & A 
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